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Abstract 


Modern,  analytical  tools  are  critical  to  understand  the  impact  of  open 
architecture  technology  and  open  business  models  on  naval  warfighting  processes 
and  procedures.  These  tools  must  measure  the  operational  value  of  a  system  from 
an  end-user,  warfighter  perspective,  identify  areas  of  deficiencies  in  capabilities,  and 
flag  areas  for  potential  acquisitions.  One  advantage  of  examining  open  architected 
system  upgrade  options  from  a  warfighter  perspective  is  that  the  new  systems  can 
be  integrated  with  reengineered  processes  more  easily  leading  to  improved  process 
performance.  This  perspective,  using  OA  to  upgrade  existing  IWS  systems,  ensures 
that  upgrades  will  lead  to  improved  warfighting  capabilities.  Traditional  measurement 
tools  used  for  cost  analysis  cannot  calculate  the  total  value  of  upgrading  a  system  to 
support  an  improved  warfighting  capability,  particularly  the  improved  operational 
value  resulting  from  reengineering  of  warfighting  processes. 

The  Knowledge  Value  Added/Real  Options  (KVA+RO)  Valuation  Framework 
is  a  tool  designed  to  assist  decision-makers  in  making  technology  acquisitions.  This 
paper  describes  research  using  the  KVA+RO  framework  for  estimating  return  on 
investment,  in  an  open  architecture  approach,  to  upgrading  and/or  replacing  aging 
IWS  AEGIS  and  SSDS  systems.  The  results  of  the  research  indicated  that  using  the 
open  architecture  (OA)model,  in  combination  with  the  “leave  and  layer”  approach, 
was  approximately  five  times  more  valuable  than  the  current  proprietary  approach  to 
system  replacement  and  was  approximately  twice  as  valuable  as  a  complete  retrofit 
and  replace  strategy.  “Leave  and  layer”  provided  the  highest  return  on  investment 
for  replacing  the  AEGIS  system  with  the  lowest  risk.  The  ultimate  success  of  the  OA 
approach  is  dependent  on  the  ability  of  the  multiple  parties  to  system  development 
and  deployment  to  collaborate.  Collaboration,  along  with  the  tools  that  facilitate 
collaboration,  is  critical  to  the  success  of  any  of  the  OA  approaches. 

Keywords:  Return  on  Investment,  Real  Options,  AEGIS,  SSDS,  Integrated  Risk 
Management 
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I.  Executive  Summary 


The  US  Navy  (Navy)  is  transforming  traditional  business  practices  through 
Naval  Open  Architecture  (Naval  OA).  Naval  OA,  a  multi-faceted,  enterprise-wide 
business  model  and  product-line  strategy  leverages  “open”  computer  design 
principles  and  architectures.  It  expands  the  technological  open  architecture  (OA) 
model  and  taps  into  a  multiple-developer  network  to  deliver  cost-effective, 
innovative,  and  rapid/spiral  acquisition  capabilities.  In  the  migration  to  an  OA 
business  model,  billions  of  dollars  in  software  and  hardware  development 
expenditures,  along  with  subsequent  maintenance  costs,  are  at  stake. 

PEO  IWS  tasked  a  research  team  from  the  Naval  Postgraduate  School  (NPS) 
to  develop  a  methodology  for  estimating  return  on  investment  (ROI)  using  an  OA 
approach  to  upgrading  and/or  replacing  the  aging  Integrated  Weapons  Systems 
(IWS)  AEGIS  and  SSDS  systems.  The  methodology  also  had  to  be  capable  of 
estimating  total  value  of  strategic  alternative  options  for  replacing  existing  AEGIS 
functionality. 

Approaching  the  project  from  a  customer-based,  warfighter  perspective,  the 
NPS  team  applied  the  Knowledge  Value  Added/Real  Options  (KVA+RO) 
valuation/risk  portfolio  management  framework  to  reengineering  situational 
awareness  (SA)  procedures  used  in  the  AEGIS  and  SSDS  platforms.1  Track 
management  sub-processes  used  in  SA  procedures  were  analyzed  through  the  KVA 
process  reengineering  methodology  under  “As  Is,”  “To  Be,”  “Radical  1,”  and  Radical 
2”  scenarios.  ROI  metrics  on  individual  sub-processes  and  watch  stations  for 
AEGIS  and  SSDS  were  generated  through  KVA,  with  a  particular  focus  on  systems 
interoperability.  ROI  estimates  reached  as  high  as  404%  for  AEGIS  and  399%  for 
SSDS. 


1 


Although  the  total  functionalities  of  AEGIS  and  SSDS  IWS  systems  are  so  broad,  we  focused  on  situational  awareness 
because  it  is  the  most  promising  area  for  upgrading  and  reengineering. 
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Real  options  analysis  was  then  performed  to  determine  the  prospective  value 
of  upgrading  the  AEGIS  IWS  over  a  nine-year  period  from  KVA  data  inputs.  Three 
options  of  “Strategy  A:  As  Is”  (i.e. ,  maintain  the  existing  proprietary  approach), 
“Strategy  B:  DDX  OA — Develop  and  Retrofit”  (i.e.,  develop  a  complete  system  using 
an  OA  approach  and  replace  the  existing  AEGIS  system),  and  “Strategy  C:  Aegis 
OA — Leave  and  Layer”  (i.e.,  use  an  OA  approach  and  replace  AEGIS  modules  over 
time)  represent  potential  system  development  and  deplolyment  strategies;  each  a 
unique  path  with  risks  and  benefits.  Real  Options  values  ranged  from  $12  billion  to 
$58.8  billion  for  the  strategic  choices. 

A.  The  KVA+RO  Framework 

KVA+RO  is  a  comprehensive  measurement  process  and  an  integrated  tool 
set  that  defines,  measures  and  evaluates  the  total  value  of  given  IWS  acquisitions. 

It  captures  data  across  a  spectrum  of  organizations  to  compare  returns  on 
investments,  outputs,  processes,  capabilities,  risks,  strategic  alternatives,  costs,  and 
value  (i.e.,  comparable  revenue).  KVA+RO  analytically  quantifies  uncertainty  and 
risks  elements  inherent  in  predicting  the  future,  includes  ways  to  mitigate  these  risks 
through  strategic  options  with  analysis  of  alternatives,  and  by  analytically  developing 
and  allocating  budgets  to  optimize  project  portfolios. 

1.  Knowledge-based  Metrics:  Knowledge  Value  Added  (KVA) 

KVA  measures  the  value  provided  by  human  capital  assets  and  IT  assets  by 
an  organization,  process  or  function  at  the  sub-process  level.  Using  a  “market 
comparables”  valuation  technique,  it  monetizes  the  outputs  of  all  assets,  including 
intangible  knowledge  assets.  Using  market  comparables  provides  a  means  for 
valuing  the  outputs  of  warfighting  processes  in  the  common  units  of  money.  This,  in 
turn,  makes  it  possible  to  use  powerful  financial  metrics  in  forecasting  the  value  of 
various  strategic  options  for  replacing  aging  IWS  systems. 

Capturing  the  value  embedded  in  an  organization’s  core  processes, 
employees  and  IT  enables  the  actual  cost  and  revenue  of  a  product  or  service  to  be 
calculated.  Analyses  like  ROI  on  individual  projects,  programs,  processes  and  sub- 


processes  within  a  portfolio  of  IT  acquisitions  can  be  derived  through  the  KVA 
methodology. 

2.  Risk  Analysis:  Real  Options  (RO) 

Potential  strategic  investments  can  then  be  evaluated  with  real  options 
analysis  based  on  KVA  data.  The  analysis  applied  is  a  robust  and  analytical  process 
incorporating  the  risk  identification  (applying  various  sensitivity  techniques),  risk 
quantification  (applying  Monte  Carlo  simulation),  risk  valuation  (applying  real  options 
analysis),  risk  mitigation  (utilizing  real  options  framing),  and  risk  diversification 
(employing  analytical  portfolio  optimization). 

3.  Study  Results  and  Recommendations  Summary 

The  results  of  our  analysis  include: 

•  The  KVA+RO  valuation  framework,  a  viable  methodology  for 
estimating  ROIs  and  projecting  valuation  of  acquisition  options, 
should  be  used  across  the  board.  Several  Department  of  Defense 
projects  are  implementing  the  framework.  The  methodology  also 
supports  the  CNO’s  recent  directive  of  accelerating  adoption  of  open- 
business  models  and  providing  a  methodology  to  assess  the  business 
risks  and  benefits  of  various  OA-based  acquisition  strategies. 

•  Upgrading  existing  IWS  functionality  to  support  reengineering 
elements  of  existing  track-management  process  appears 
beneficial.  ROIs  ranged  from  212%  to  404%  for  the  AEGIS  platform 
and  ROIs  for  the  SSDS  platform  were  also  significant.  ROI  results  are 
shown  in  Figures  1  and  2.2  In  addition,  Tables  1  and  2  provide  a 
detailed  analysis  of  the  ROIs  for  reengineering  the  track  management 
process.  Table  3  summarizes  the  reengineered  processes  and 


2  Radical  1  scenario  assumes  the  improvements  of  the  “To  Be”  scenario  while  the  Radical  2  scenario  assumes  cumulative 
improvements  from  all  three  scenarios. 


subsequent  benefits.  The  results  are  based  on  the  assumption  that  the 
IWS  systems  could  be  developed  within  an  OA  framework  to  support 
the  reengineered  process  designs. 


Figure  1.  KVA  Results:  AEGIS  ROI  Estimates 


Figure  2.  KVA  Results:  SSDS  ROI  Estimates 


Tables  1  and  2  are  more  detailed  results  for  ROI  analysis  for  “As  Is”  and  the  other  three 
increasingly  automated  scenarios. 
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Table  1.  Detailed  ROI  Estimates  for  AEGIS 


AS  IS 

TO  BE 

RAD  1 

RAD  2 

CORRELATE 

Obtain  Link  Information 

3421% 

3307% 

3063% 

2633% 

Identify  "Same  Contact,  Multiple  Track" 

-91% 

-91% 

2061% 

1756% 

Verify  Other  Track  Sources 

-95% 

-95% 

-95% 

-96% 

Correlate  sub-total 

1184% 

1141% 

1506% 

1296% 

TRACK 

Monitor  Suspect  Tracks 

-98% 

-99% 

-99% 

-99% 

Update  Tracks 

-97% 

-97% 

361% 

310% 

Update  GCCS-M 

-97% 

91% 

84% 

69% 

Track  sub-total 

-98% 

-94% 

-58% 

-64% 

IDENTIFY 

Verify  IFF  signal 

802% 

769% 

706% 

607% 

Verify  EW  emissions 

-91% 

-91% 

-92% 

509% 

Verify  Point  of  Origin 

-98% 

4121% 

3821% 

3332% 

Match  Against  ATO 

-98% 

4206% 

3890% 

3382% 

Match  Against  CommAir  Profile 

863% 

835% 

763% 

643% 

Match  Against  Intel  Information 

-97% 

-97% 

-97% 

3814% 

Examine  Kinematic  Data 

-96% 

-96% 

-97% 

-97% 

Obtain  Visual  ID 

-100% 

-100% 

-100% 

-100% 

Conduct  Verbal  Query 

-99% 

-99% 

-99% 

-99% 

Identify  sub-total 

8% 

60% 

50% 

326% 

RELAY 

Send  Over  Links 

-87% 

-88% 

-89% 

-90% 

Discuss  Picture  with  Battle  Force  Units 

-98% 

-99% 

-99% 

-99% 

Relay  sub-total 

-97% 

-98% 

-98% 

-99% 

Totals 

212% 

240% 

273% 

404% 
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Table  2.  Detailed  ROI  Estimates  for  SSDS 


AS  IS 

TO  BE 

RAD  1 

RAD  2 

CORRELATE 

Obtain  Link  Information 

3393% 

3280% 

3026% 

2598% 

Identify  "Same  Contact,  Multiple  Track" 

-91% 

-91% 

2530% 

2158% 

Verify  Other  Track  Sources 

-95% 

-95% 

-96% 

-96% 

Correlate  sub-total 

1174% 

1131% 

1512% 

1301% 

TRACK 

Monitor  Suspect  Tracks 

-98% 

-99% 

-99% 

-99% 

Update  Tracks 

-96% 

-97% 

546% 

475% 

Update  GCCS-M 

-98% 

14% 

10% 

1% 

Track  sub-total 

-98% 

-96% 

-53% 

-60% 

IDENTIFY 

Verify  IFF  signal 

790% 

757% 

692% 

595% 

Verify  EW  emissions 

-90% 

-91% 

-91% 

474% 

Verify  Point  of  Origin 

-98% 

3689% 

3405% 

2967% 

Match  Against  ATO 

-98% 

3813% 

3510% 

3049% 

Match  Against  CommAir  Profile 

926% 

896% 

816% 

688% 

Match  Against  Intel  Information 

-97% 

-97% 

-97% 

3688% 

Examine  Kinematic  Data 

-96% 

-96% 

-96% 

-96% 

Obtain  Visual  ID 

-100% 

-100% 

-100% 

-100% 

Conduct  Verbal  Query 

-99% 

-99% 

-99% 

-99% 

Identify  sub-total 

12% 

59% 

48% 

316% 

RELAY 

Send  Over  Links 

-82% 

-83% 

-84% 

-86% 

Discuss  Picture  with  Battle  Force  Units 

-99% 

-99% 

-99% 

-99% 

Relay  sub-total 

-97% 

-98% 

-98% 

-98% 

Totals 

212% 

237% 

271% 

399% 

Table  3  discusses  the  potential  impact  of  an  OA  approach  on  AEGIS  and 

SSDS. 

Table  3.  Potential  Benefits  of  OA  Combined  with  Reengineering  of  Track 

Management  Operations 


“As  Is” 

“To  Be” 

“Radical  1” 

“Radical  2” 

None 

Limited  re-engineering 

Significant  re-engineering 

Substantial  re-engineering 

Technology 

Impact 

None 

•  Info  provided  in  ATO  could  be 
upgraded  into  AEGIS, 
reducing  manpower 
requirements. 

•  Enables  greater  sensor  and 
data  integration,  providing 
enhanced  correlation  in 
pinpointing  origin  of  aircraft  or 
ship. 

•  Streamlined  system  automatically 
updates  tracks. 

•  Increased  information-sharing  and 
collaborative  technology  allows  for 
automatic  correction  of  multiple  tracks 
per  target. 

•  Continuously  updates  tracks,  allowing 
for  pinpoint  accuracy. 

•  Collaborative  technology  minimizes 
possibility  of  multiple  tracking  of 
targets. 

•  Includes  changes  from  “To  Be” 

•  Collaborative  technology  automatically 
updates  ship’s  systems  with  Intel 
information. 

•  Electronic  communication  of  data  from  EW 
to  CIC  personnel  facilitates  COTS-based 
environment  that  easily  upgrades  to 
accommodate  greater  processor  speeds. 

•  Greatly  enhances  CIC  efficiency  through 
more  timely  SA. 

•  Includes  changes  from  “To  Be”  and 
“Radical  1 .” 

Potential 

Benefits* 

•  Reduces  maintenance  costs. 

•  Frees  watch-standers  to 
perform  other  tasks  while 
providing  faster  data  flow. 

•  Increases  accuracy  of  tracking  targets. 

•  Substantial  re-engineering  leads  to  drastic 
reduction  in  watch-stander  work  time, 
greatly  reducing  human  error  and  further 
decreasing  maintenance  costs. 
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•  Strategy  C:  Leave  and  Layer  is  the  most  promising  strategy  with 
lowest  total  costs.  It  has  the  highest  potential  rate  of  return  with  a 
valuation  of  $58.8  billion  and  4.9  times  the  potential  return  than  Strategy  A. 
Strategy  B  has  a  valuation  of  $23.2  billion,  while  Strategy  A  has  the  lowest 
valuation  at  $12  billion. 

Table  4.  Real  Options  Valuation  Results:  Strategies  A-C 


Strategy  A 

Strategy  B 

Strategy  C 

STRATEGIC  OPTION 

“As  Is” 

DDX  OA 

“Develop  and  Retrofit” 

AEGIS  OA 
“Leave  &  Layer” 

Net  Present  Value 

$12.00B 

$6.38B 

$27.52B 

Real  Options  Value 

$12.00B 

$23.16B 

$58.84B 

Total  Cost 

$10.00B 

$24.00B 

$9.09B 

Strategic  Real  Options- 
based  Relative  Return  Ratio 

1.00 

1.9 

4.9 

•  Collaboration  is  critical.  OA  as  an  acquisition,  development  and 
deployment  framework  will  not  succeed  without  the  support  of  a 
collaborative  infrastructure  to  facilitate  the  introduction  of  multiple 
large,  medium  and  smaller  players  and  their  necessary  interactions 
with  users  of  the  systems  (e.g.,  warfighters),  the  acquisition 
community,  and  Navy  leadership.  Significant  investments  will  be 
required  for  the  infrastructure  necessary  to  enable  all  parties 
(acquirers,  users,  developers)  to  collaborate  easily  and  effectively  in  an 
OA  model. 

•  Performance  monitoring  is  required.  If  the  performance  of 
acquisition  strategies  is  not  monitored  overtime,  the  probability  of 
success  will  be  greatly  reduced.  Performance  measurement  systems 
(i.e.  performance  accounting  software),  in  conjunction  with  predictive 
forecasting  software  programs,  provide  additional  analytic  support  to 
IWS  systems-acquisition  strategies. 
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These  research  results,  along  with  components  of  the  KVA+RO  framework 
and  key  findings  from  the  analysis,  are  summarized  in  this  report. 3 

B.  Summary 

IWS  systems  developed  in  a  closed,  proprietary  model  have  performed  well 
and  provide  substantial  returns.  However,  a  new  paradigm  is  required  to  maintain 
military  superiority  and  wage  information-age  warfare.  Through  open  architected 
system  development  and  open-business  models,  benefits  such  as  reusable  code, 
lower  maintenance-upgrade  costs,  and  greater  vendor  flexibility  in  supporting 
system  module  upgrades  could  be  derived.  Moreover,  the  Navy  can  leverage  new 
technology  by  quickly  adopting  it  to  warfighter  needs. 

This  study  found  that  the  “leave  and  layer”  option  for  IWS  replacement 
provided  the  lowest  costs,  highest  ROIs,  and  highest  strategic  options  value  with  the 
lowest  risk.  The  results  recommend  use  of  the  OA  -  leave  and  layer  IWS 
replacement  approach  to  support  reengineered  warfighting  processes. 


The  accuracy  of  our  analysis  is  dependent  on  data  and  information  provided  by  subject-matter  experts.  KVA  analysis 
includes  tests  of  the  reliability  of  their  estimates. 


II.  Introduction 


Naval  Open  Architecture  (Naval  OA)  is  a  multi-faceted,  enterprise-wide 
business  model  and  product-line  strategy  designed  to  fully  capitalize  on  “open” 
computer  design  principles  and  architectures.  Expanding  on  the  technological  open 
architecture  (OA)  model,  Naval  OA  leverages,  “open  business  models  for  the 
acquisition  and  spiral  development  of  new  systems  that  enable  multiple  developers 
to  collectively  and  competitively  participate  in  cost-effective  and  innovative  capability 
delivery  to  the  Naval  Enterprise”  (Mullen,  2006,  August  28).  The  new  OA  business 
model  requires  a  greater  degree  of  collaboration  among  customers  (e.g.,  warfighter), 
builders  (e.g.,  small,  medium,  and  large  technology  companies),  and  buyers  (e.g., 
the  acquisition  community)  than  the  existing  closed,  proprietary  IWS  business 
model.  In  the  migration  to  an  OA  business  model,  billions  of  dollars  in  hardware 
and  software  development  expenditures,  along  with  subsequent  maintenance  costs, 
are  at  stake. 

To  understand  the  potential  impact  of  OA  technology  and  business  models  on 
naval  warfighting  processes  and  procedures,  analytical  tools  are  critical  for  decision¬ 
makers  as  they  manage  their  portfolio  of  options.  Portfolio  management  requires 
that  these  tools  quantify  the  risks,  costs,  and  net  value  of  potential  IWS  acquisitions. 
The  tools  must  be  able  to  help  identify  where  gaps  exist  in  current  processes  and  to 
project  anticipated  returns  on  investments  to  fill  those  gaps. 

This  study  describes  research  conducted  at  the  Naval  Postgraduate  School 
(NPS)  using  the  Knowledge  Value  Added/Real  Options  (KVA+RO)  valuation/risk 
portfolio  management  framework.  KVA+RO  is  a  comprehensive  measurement  tool 
set  that  defines,  measures  and  evaluates  total  value  of  given  IWS  acquisitions.  It 
captures  data  across  a  spectrum  of  organizations  to  compare  outputs,  processes, 
capabilities,  risks,  costs,  and  value  (i.e.  comparable  revenue).  KVA+RO  analytically 
quantifies  uncertainty  and  risk  elements  inherent  in  predicting  the  future,  includes 
ways  to  mitigate  these  risks  through  strategic  options  and  by  analytically  developing 
and  allocating  budgets  to  optimize  project  portfolios.  Understanding  uncertainties 


and  mitigating  the  potential  impact  of  risks  significantly  improves  the  likelihood  of 
successful  acquisition  decisions. 

In  this  study,  KVA+RO  is  used  to  assess  the  implications  of  OA  on  SA 
procedures  onboard  the  AEGIS  and  SSDS  platforms.  Focusing  on  systems 
interoperability,  KVA  methodology  is  first  applied  to  generate  knowledge-based,  ROI 
metrics  on  individual  sub-processes  and  watch  stations  involved  in  track- 
management  processes.  The  potential  impact  of  OA  on  track  management 
processes  and  sub-processes  is  analyzed  under  several  scenarios:  “To  Be,” 
“Radical  1,”  and  “Radical  2” for  AEGIS  and  SSDS.  Potential  investments  are  then 
evaluated  for  AEGIS  through  real  options  analysis,  resulting  in  net  present  value 
(NPV)  of  three  strategic  alternatives  ranging  from  $6.4  billion  to  $27.5  billion  over  a 
nine-year  period  and  options  valuations  of  from  $23.2  to  $58.8  billion. 
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III.  Lessons  from  the  “Open”  Solutions  Movement 


Disruptive  forces  and  accelerating  shifts  in  technology  have  enabled 
organizations  to  leverage  open  technology  platforms  to  achieve  greater  productivity 
and  efficiency  levels.  These  “open”  solutions  offer  new  possibilities  for  solving 
business  problems,  provide  business  interoperability  by  standardization  and 
technology  transparency,  and  decrease  time  to  market  for  key  products  and 
services.  Organizations  are  adopting  open  technology  platforms  and  open-source 
software  for  critical  business  needs,  moving  into  mainstream  business  practices  in 
corporations  such  as  IBM,  Google,  Intel,  JPMorgan  Chase,  Merrill  Lynch  and  Pfizer. 

One  manifestation  of  the  movement  toward  “openness,”  yet  to  be  embraced 
by  the  Department  of  Defense  (DoD),  is  the  open-source  software  movement. 
Germany,  Australia,  the  United  Kingdom,  Finland,  Norway,  Canada,  China,  Japan 
and  Brazil  are  among  the  increasing  number  of  governments  embracing  open- 
source  software.  Open-source  software  is  growing  at  such  a  rate  that  it  represents 
the  most  significant  all-encompassing  and  long-term  trend  that  the  software  industry 
has  seen  since  the  early  1980s,  according  to  a  recent  study  by  International  Data 
Corporation  (IDC).  IDC’s  survey  of  over  5,000  developers  in  1 16  countries  found 
that  open  source  software  is  being  used  by  71%  of  developers  in  the  world  and  is  in 
production  at  54%  of  their  organizations. 

Open-source  software  development  site  SourceForge.net  reported  more  than 
129,000  projects  in  2006,  up  from  1,362  projects  in  2000.  Google’s  2006  “Summer 
of  Code”  open-source  initiative  has  630  collaborative  projects  pumping  more  than  $3 
million  back  into  the  open-source  community.  There  are  now  more  than  55  Open- 
Source  Initiative  (OSI)  certified  open-source  licenses  available  given  the  popularity 
of  open-source  software.4  The  success  of  this  movement  has  been  predicated  on 


4  According  to  OSI,  the  most  commonly  used  licenses  are:  Apache  License,  GNU  General  Public  License  (GPL),  GNU  Lesser 
General  Public  License  (LGPL),  Modified  BSD  (Berkeley  Software  Distribution)  License  (new  BSD)  and  Mozilla  Public 
License  (MPL).  MPL  is  the  most  widely  used  since  1998.  A  NASA  license  is  also  available  (OSI,  2006). 
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the  ability  of  the  multiple  parties  involved  to  easily  collaborate  across  organizational, 
field-specific,  and  national  boundaries.  While  this  approach  to  software 
development  may  not  directly  apply  to  security-sensitive  systems  such  as  IWS, 
lessons  can  be  learned  by  examining  the  results  of  this  movement  in  the  commercial 
world. 


A.  Collaboration  is  Key 

As  with  the  use  of  the  OA  technology  and  business  model,  open-source  is 
built  on  the  tenants  of  open  access  and  collaboration.  The  lessons  learned  when 
“openness”  is  applied  to  system  development  and  business  models  from  the  open- 
source  movement  is  that  such  approaches  allow  access  to  a  wider  development 
community  that  can  adapt,  improve  and  fix  software  at  a  faster  and  more  agile  pace 
than  can  a  proprietary  vendor.  Organizations  are  also  not  locked-into  one  vendor  or 
product. 

Google,  for  example,  has  acknowledged  that  the  open  architecture  of  Google 
Maps,  allowing  external  developers  to  build  applications  on  top  of  it,  greatly 
contributed  to  the  mapping  service’s  functionality  and  diversity  at  a  greater  level  than 
the  company  could  have  done  internally  (Perez,  2006,  March  6).  In  2005,  IBM 
opened  access  to  500  corporate  software  patents,  forfeiting  $10  million  dollars  in 
annual  royalties.  According  to  IBM,  technological  advances  are  often  dependent  on 
shared  knowledge,  standards  and  collaborative  innovation  (IBM,  2005,  January  11). 
IBM  believes  that  by  being  allowed  access  to  those  patents,  open-source  developers 
will  help  foster  continued  innovation. 

It’s  critical  to  note  that  the  full  potential  of  OA  and  open-business  model 
approaches  cannot  be  achieved  without  a  basic  collaborative  technology 
infrastructure.  The  ease  with  which  all  parties  share  ideas,  compare  requirements, 
develop  solutions,  test  system  capabilities,  and  finally  participate  jointly  in  deploying 
systems  is  dependent  on  their  commitment  to  collaborate  openly  and  fully  with  each 
other.  This  can  only  be  facilitated,  realistically  and  practically,  through  collaborative 
technology.  Benefits  from  this  type  of  approach  have  been  previously  demonstrated 


-4- 


in  shipyard  planning  where  the  use  of  product  lifecycle  management  (PLM) 
collaborative  software  added  tremendous  potential  value  to  the  process  (Komoroski, 
C.,  Housel,  T.,  Horn,  S.,  &  Mun,  J.,  2006,  October). 

B.  Open  Architecture  and  the  Department  of  Defense 

Computer  software  plays  a  critical  component  in  maintaining  the  nation’s 
defenses.  For  example,  less  than  10%  of  its  functionality  was  provided  by  software 
when  the  F-4  fighter  was  developed  in  the  1960’s;  at  least  80%  of  the  F/A-22’s 
functionality  is  software  related  (GAO,  2004,  March).  Although  the  DoD  spends 
billions  of  dollars  to  develop  and  maintain  rights  to  millions  of  lines  of  code,  such 
software  cannot  be  accessed  or  modified  by  anyone  but  the  original  vendor  because 
of  its  proprietary  nature  (Payton,  2006,  August  14).  Moreover,  the  DoD  will  spend 
as  much  as  $12  billion  on  reworking  software  for  major  weapons  acquisitions 
programs — 30%  of  its  estimated  budget  of  $40  billion  for  research,  development, 
testing  and  evaluation  in  Fiscal  Year  2006  (Wait,  2006,  July  3).  Consequences 
resulting  from  the  lack  of  OA  and  open  business  models  include: 

•  increased  development  and  maintenance  costs  for  information  technology; 

•  lock-in  to  obsolete  proprietary  technologies; 

•  inability  to  extend  existing  capabilities  in  months  versus  years;  and 

•  lack  of  interoperability  due  to  opacity  and  stove-piping  of  information  systems. 

(Herz,  J.C.,  Lucas,  M.,  &  Scott,  J.,  2006) 

The  DoD  has  at  least  1 15  open-source  software  applications  used  in  more 
than  250  applications.  However,  IWS  software  acquisitions  are  still  made  with  the 
same  industrial-age  business  models  used  to  acquire  ships,  tanks  and  other 
physical  machinery  (Payton,  2006,  August  14).  The  traditional  business  model  of 
purchasing  physical  goods  and  services  falls  short  when  applied  to  acquiring  digital 
assets  like  IWS  technology.  New  business  models  are  required  to  acquire  IWS 
technology  to  wage  information-age  warfare  requiring  responsiveness  and  agility, 
according  the  Deputy  Under  Secretary  of  Defense  for  Advanced  Systems  and 
Concepts,  Dr.  Sue  Payton  (Payton,  2006,  August  14).  Moving  to  an  open 
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architecture  model  maximizes  IT  acquisitions  by  saving  development  dollars, 
reducing  development  cycles,  and  fostering  new  and  innovative  solutions  and 
capabilities. 

Modern,  analytical  tools  are  also  necessary  to  deploy  an  open  solutions 
business  strategy.  These  tools  must  measure  the  operational  value  of  a  system 
from  the  warfighter’s  perspective,  identify  areas  of  deficiencies  in  capabilities,  and 
flag  areas  for  process  improvement.  Traditional  measurement  tools  used  for  cost 
analysis  cannot  calculate  the  total  value  of  a  system,  particularly  operational  value 
provided  by  specific  process  improvements.  At  the  tactical  level,  an  operator  does 
not  define  capabilities  merely  in  cost  terms  but  also  in  time,  efficiency  and 
effectiveness  gains  like  processing  more  targets  within  a  given  time  period.  Given 
new  potential  threats,  such  as  “swarm”  attacks  where  there  may  be  thousands  of 
targets  at  any  one  time,  the  warfighter’s  perspective  in  developing  open  and  agile 
systems  is  critical.  Focusing  on  the  potential  cost  reductions  from  the  OA  and  open 
business  models  approach  may  lead  developers  and  acquirers  away  from  the  real 
needs  of  the  warfighter.  Maintaining  a  focus  on  the  potential  value  produced,  in 
addition  to  potential  cost  savings,  is  critical  to  the  success  of  OA  and  open  business 
models. 
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IV.  KVA+RO  Framework 


The  KVA+RO  methodology  provides  an  equal  focus  on  the  potential  value 
and  cost  of  new  IWS  systems.  KVA+RO  measures  operating  performance,  cost- 
effectiveness,  return  on  investments,  risk,  real  options  (capturing  strategic  flexibility), 
and  analytical  portfolio  optimization.  In  this  study,  it  was  applied  to  the  problem  of 
finding  the  most  promising  solution  for  replacing  aging  IWS  systems  such  as  AEGIS 
and  SSDS  to  support  reengineered  warfighting  capabilities.  KVA+RO  analysis 
empowers  decision-makers  and  supports  IWS  acquisition  strategies  by  providing 
performance-based  data  and  scenario  analysis.  Analyses  like  ROI  on  individual 
IWS  projects  and  programs,  as  well  as  processes  and  sub-processes  (e.g.,  track 
management  processes  in  the  present  study)  supported  by  IWS  systems  can  be 
examined  within  a  portfolio  of  acquisitions  framed  through  the  KVA+RO 
methodology.  An  overview  of  the  framework  is  shown  in  Figure  1 . 
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Figure  1.  NPS  Valuation  Framework 
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The  framework  has  been  used  in  a  variety  of  NPS  analyses,  including 
evaluating  the  potential  impact  of  commercial-off-the-shelf  (COTS)  technology  on 
naval  maintenance/modernization  processes.  In  the  study  involving  one  specific 
area  of  shipyard  planning  for  maintenance  alterations,  cost  savings  were  projected 
to  exceed  $40  million  per  year  and  manpower  requirements  drastically  reduced  with 
commercial-off-the-shelf,  three-dimensional  scanning/visualization  technology  and 
collaborative  PLM  technology  (Komoroski,  C.,  Housel,  T.,  Horn,  S.,  &  Mun,  J.,  2006, 
October).  Key  components  of  the  NPS  Valuation  framework  are  further  discussed  in 
this  section. 
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A.  Knowledge-based  Metrics:  Knowledge  Value  Added 
(KVA) 

KVA  measures  the  value  provided  by  human  capital  assets  and  IT  assets 
(e.g.,  IWS  systems  +  human  operators)  by  an  organization,  process  or  function  at 
the  sub-process  level.  It  monetizes  the  outputs  of  all  assets,  including  intangible 
knowledge  assets.  Capturing  the  value  embedded  in  an  organization’s  core 
processes,  employees  and  IT  enables  the  actual  cost  and  revenue  of  a  product  or 
service  to  be  calculated.  Figure  2  identifies  the  types  of  assets  used  to  produce 
output;  outputs  can  be  products  or  services  produced  by  that  organization. 

Figure  2.  Measuring  Output 


Figure  3  shows  how  KVA  process  costing  differs  from  traditional  accounting 
methods. 


-9- 


Figure  3.  Comparison  of  Traditional  Accounting  versus  Process-based 
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As  seen  in  Table  1 ,  total  value  is  captured  in  two  key  metrics:  ROI  and  ROK. 
While  ROI  is  the  traditional  financial  ratio,  ROK  identifies  how  a  specific  process 
converts  existing  knowledge  into  process  outputs  so  decision-makers  can  quantify 
costs  and  measure  value  derived  from  investments  in  productive  assets.  A  higher 
ROK  signifies  better  utilization  of  knowledge  assets.  If  IT  investments,  such  as 
existing  IWS  systems,  do  not  improve  the  ROK  value  of  a  given  process,  steps  must 
be  taken  to  improve  that  process’s  function  and  performance. 

Table  1.  KVA  Metrics 


Metric 

Description 

Type 

Calculation 

Return  on  Knowledge 
(ROK) 

Basic  productivity,  cash¬ 
flow  ratio 

Sub-corporate,  process-level 
performance  ratio 

Outouts-Benefits  in  Common  Units 

Cost  to  Produce  Output 

Return  on  Investment 
(ROI) 

Same  as  ROI  at  the  sub¬ 
corporate,  process  level 

Traditional  investment 
finance  ratio 

(Revenue-Investment  Cost) 

Investment  cost 

B.  Risk  Analysis:  Real  Options  (RO) 

Potential  strategic  investments  can  then  be  evaluated  with  real  options 
analysis  based  on  KVA  data.  This  analysis  is  a  robust  and  analytical  process 
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incorporating  the  risk  identification  (applying  various  sensitivity  techniques),  risk 
quantification  (applying  risk-based  Monte  Carlo  simulation),  risk  valuation  (applying 
real  options  analysis),  risk  mitigation  (utilizing  real  options  framing),  and  risk 
diversification  (employing  analytical  portfolio  optimization)  using  the  Real  Options 
SLS  and  Risk  Simulator  software  programs.  Figure  4  reflects  the  complex 
calculations  for  integrated  risk  analysis  in  KVA+RO. 

Figure  4.  Integrated  Risk  Analysis 


1.  List  of  projects  and 
strategies  to  evaluate 


2.  Base  case  projections  for  each 
project 

Historical  Data  Gathering 
Time  Series  Forecasting 
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models  with  KVA  data 


Start  with  a  list  of  projects  or 
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projects  have  already  been 
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5.  Framing  Real  Options 


will  the  assistance  of  time- 
series  forecasting  and 
historical  data. . . 


6.  Options  analytics,  simulation 
and  optimization 
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4.  Dynamic  Monte  Carlo 
simulation 


Simulation 

Lognormal 


series  of  static  base  case  financial 
(discounted  cash  flow)  models  for 
each  project. . . 

7.  Portfolio  optimization  and  asset 
allocation 


. .  .sensitivity  and  scenario  analysis 
coupled  with  Monte  Carlo  simulation  is 
added  to  the  analysis  and  the  financial 
model  outputs  become  inputs  into  the  real 
options  analysis _ 


8.  Reports  presentation  and 
update  analysis 


. .  .the  relevant  projects  are 
chosen  for  real  options 
analysis  and  the  project  or 
portfolio  real  options  are 
framed. . . 


Closed-Form  Models 
^ —  =  /J  (£  t )  +  os  -JS  t 


. .  .real  options  analytics  are  calculated 
through  binomial  lattices  and  closed-form 
partial -differential  models  with 
simulation. . . 


. .  .stochastic 
optional  step  if 


iptimization  is  the  next 
lultiple  projects  exist  that 
requires  efficient  asset  allocation  given 
some  budgetary  constraints. . .  useful  for 
strategic  portfolio  management. . . 


..create  reports,  make  decisions, 
""and  do  it  all  again  iteratively  over 
time. . . 


C.  Beyond  Concept:  KVA+RO  Implementations 

Moving  beyond  a  concept  stage,  the  KVA+RO  framework  is  being 
implemented  in  SPAWAR  and  in  the  Army  Rapid  Equipping  Force  project.  KVA+RO 
is  being  used  in  both  projects  to  improve  processes,  reduce  cycle-times  and  costs, 
and  increase  value  by  allowing  Navy  executives  to  acquire  intelligence  systems  via 
a  portfolio  approach  and  by  getting  the  Army  troops  in  the  field  (i.e.,  Iraq  and 
Afghanistan)  what  they  need  very  quickly  through  new  rapid  acquisition  processes. 
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V.  Proof-of-Concept  Case  Study:  Situational 
Awareness  Onboard  AEGIS  and  SSDS  Platforms5 


This  proof  of  concept  case  study  is  designed  to  assist  PEO  IWS,  Open 
Architecture  Division,  with  its  mandate  of  implementing  OA  in  the  Navy.  The  case  is 
prepared  from  a  warfighter  perspective  because  the  value  of  OA  must  be  proven  to 
the  ultimate  end-user.  This  perspective  also  permits  a  review  of  how  OA  can  lead  to 
flexible  system  acquisition  and  development  to  enable  reengineered  processes  that 
will  provide  better  performance  in  core  warfighting  processes  such  as  SA. 

In  a  multi-phased  approach,  KVA+RO  was  applied  to  SA-track  management 
procedures  used  in  the  AEGIS  and  SSDS  platforms.  As  illustrated  in  Figure  5,  the 
total  functionalities  of  AEGIS  and  SSDS  systems  are  very  broad  so  we  focused  our 
research  on  SA  because  it  appeared  to  be  the  most  promising  area  for  upgrading 
and  reengineering  according  to  subject  matter  experts.  The  goals  of  this  research 
were  to: 

•  Demonstrate  the  efficacy  of  the  KVA-RO  framework  to  evaluate 
reengineering  designs  for  warfighting  core  processes  (i.e.,  SA-track 
management)  in  terms  of  the  ROI  and  strategic  option  value  of  various 
OA  approaches  to  replacing  aging  IWS  systems. 

•  Determine  which  elements  of  the  track  management  process  could  be 
reengineered  using  an  OA  approach 

•  Identify  areas  of  improvement  for  current  surface  ship  track- 
management  processes  using  the  existing  two  IWS  systems: 
AEGIS  and  SSDS. 


5  Information  collected  from  subject-matter  experts  (SMEs)  from  Surface  Warfare  Fleet  and  training  commands  at  Dahlgren 
(AEGIS)  and  Wallops  Island  (SSDS).  Information  gathered  from  SMEs  then  aggregated  to  provide  an  average  for  each 
process  to  ensure  accuracy.  Additional  information  collected,  including  process  flow  diagrams,  use-case  diagrams  and 
iiterature  review,  to  develop  baseline  data. 
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Figure  5.  Planned  Reuse  of  Aegis  and  SSDS  in  DD(X) 


Aegis 


[  I  SNSC2  Capability  08  (in  progress)  Bl  Capability  10  (Proposed)  I  I  Capability  12  (Proposed) 


Source:  MacRitchie,  J.,  “Open  Architecture  &  SNSC2,”  Presentation  June  29,  2005,  p.  12. 

A.  Background 

In  the  late  1990s,  the  DoD  articulated  a  vision  for  network-centric  (or  “net- 
centric”)  warfare  in  which  networking  military  forces  would  facilitate  information 
sharing  and  collaboration,  leading  to  enhanced  SA.  Information  superiority  is  vital  to 
enhanced  SA,  rapid  decision-making,  improved  efficiency,  speedy  execution  and 
mission  effectiveness.  A  high  degree  of  interoperability  is  required  to  achieve 
information  superiority.6  Lack  of  interoperability  between  the  services  makes  it 
difficult  for  the  warfighter  to  distinguish  “friend”  from  “foe”  and  to  make  critical 
decisions — potentially  delaying  military  response  times  or  contributing  to  lethal 
mistakes.  Figure  6  shows  a  scenario  in  which  a  sea-based  system  and  a  land- 


6  The  DoD  defines  interoperability  as  the  ability  of  systems,  units,  or  forces  to  exchange  data,  information,  materiel,  and 
services  to  enable  them  to  operate  effectively  together. 
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based  system  are  tracking  aircraft  and  are  unable  to  integrate  their  views  of  a 
battlefield. 


Figure  6.  Scenario  for  Tracking  Threats  without  Benefit  of  Interoperable 

Systems 


Enemy 

aircraft 


\ 


\ 

\ 


What  Sea- Based  System  Sees 

Correcfly  detects  one  enemy 
aircraft  and  one  friendly  aircraft 

_ * _ 3^ j 

rm 

^  \  1 

cannot  talk  to 
each  other 
directly  to 
resolve 
discrepancies 


What  Land-Based  System  Sees 


Detects  only  one  aircraft  and 
does  not  know  whether  the 
aircraft  is  friendly  or  not 


Source:  DOD  (data);  GAO  (analysis  and  presentation). 


Source:  Government  Accountability  Office.  (2006,  January).  DOD  management  approach  and  processes  not 
well-suited  to  support  development  of  global  information  grid.  Washington,  DC.  GAO  Report  GAO-06- 

211,  p.6. 


To  achieve  information  superiority  and  enable  net-centric  warfare,  the  DoD 
has  been  developing  the  Global  Information  Grid  (GIG).  Interoperable  systems  are 
critical  to  the  GIG  for  joint  military  operations  and  to  allow  users  access  to  data  on 
demand,  to  share  information  in  real-time  and  to  collaborate  in  decision-making  from 
almost  any  location.  Development  of  this  capability  is  ultimately  dependent  on 
support  from  reengineered  SA  systems  within  IWS  suites. 
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B.  Naval  Challenges 

The  Navy  must  develop  architectures  that  meet  the  integration  requirements 
for  the  GIG.  This  is  a  critical  requirement  in  designing  and  implementing  new  IWS 
systems  built  within  the  OA  framework.  In  addition,  the  Navy  must  resolve  the 
following  issues  that  are  a  result  of  legacy  technology  systems,  such  as  the  aging 
IWS  systems. 

•  Limited  computational  and  operational  capability.  Systems  operating 
at  99%  capacity  in  non-stressed  environments. 

•  Difficulty  or  inability  to  add  new  warfighting  missions.  “Stove-piped” 
systems  diminish  interoperability  and  ability  to  meet  national  security 
threats. 

•  Prohibitive  software  maintenance  costs.  Some  $3+  billion  spent 
across  Future  Years  Defense  Plan  in  PEO  IWS  to  develop  and  maintain 
computer  programs.  Additional  testing  and  certification  required  when  new 
capabilities  added. 

The  Navy  has  historically  acquired  IWS  systems  that  are  proprietary  in  design 
and  engineering,  require  unique  parts,  equipment,  and  services  to  support  them,  are 
supported  by  a  limited  number  of  suppliers,  and  become  very  expensive  to  maintain 
(Strei,  2003,  April  1).  Moreover,  systems  and/or  platforms  were  entirely  eliminated 
rather  than  upgraded  or  modernized  because  of  prohibitive  costs.  Rapid 
technological  obsolescence,  compounded  by  exorbitantly  escalating  costs  for 
proprietary  systems  are  daunting  challenges  because  design,  development,  and 
acquisition  timelines  can  span  as  much  as  15  years  before  a  military  platform 
reaches  operating  forces  (Strei,  2003,  April  1). 


C.  Naval  Open  Architecture  and  Open  Business  Models 

OA  and  open-business  models  propel  the  Navy  into  the  next  era  of  joint 
interoperability  while  resolving  legacy  issues  that  provide  new  benefits,  including: 

■  Lower  lifecycle  costs  for  IWS  systems.  Total  cost  of  ownership 
decreases  due  to  increased  maintainability,  interoperability, 
upgradeability  and  use  of  a  wider  variety  of  vendors. 
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■  Better  performing  systems.  Ability  to  rapidly  upgrade  hardware  and 
software  with  the  latest  technology  enables  greater  capabilities, 
efficiencies  and  interoperability  to  enable  reengineered  warfighting 
processes. 

■  Improved  interoperability  for  joint  warfighting.  Software  reuse  and 
modularity  facilitates  interoperability  between  systems  that  use  an 
open  architecture  framework. 

■  Facilitating  competition  and  increasing  cooperation  between 
commercial  and  military  electronics  industries.  Moving  away  from 
proprietary  systems  enables  a  broader  range  of  ideas  and 
technological  solutions. 

Guiding  principles  behind  Naval  OA  are  modularity,  reusability, 
interoperability,  lifecycle  affordability  and  collaboration  and  competition.  In  adopting 
an  open,  OA  strategy  based  on  commercially  available,  non-proprietary  information 
technology  (IT)  standards,  interfaces  and  formats,  the  Navy  will  need  to  increase 
collaboration  (e.g.,  supported  by  readily  available  collaborative  product  lifecycle 
management  technology)  to  spur  competition  and  fuel  innovation  in  the  acquisition 
lifecycle. 

Ease  of  collaboration  is  critical  to  Naval  OA  to  ensure  that  multiple  vendors 
compete,  including  the  smaller,  more  nimble  companies.  Collaboration  also 
provides  the  infrastructure  necessary  to  facilitate  all  parties  sharing  critical 
requirements  and  performance  information  to  reduce  system  modification,  re-fresh 
or  replacement  cycle-times.  As  such,  collaborative  capabilities  will  facilitate  moving 
OA  beyond  a  purely  technical  focus  to  a  more  encompassing  open-business  model, 
one  advocated  by  CNO  Admiral  Mullen.  As  noted  earlier,  Admiral  Mullen’s  vision  for 
open  architecture  isn’t  limited  to  systems  built  to  a  set  of  open  standards,  but 
focuses  on  open-business  models  tapping  into  a  multiple-developer  network  to 
deliver  innovative,  cost-effective  and  rapid,  spiral  acquisition  capabilities  to  the  Navy. 

Migrating  to  an  OA  environment  has  been  slow,  however,  despite  the  Navy’s 
early  adoption  of  open  source  strategies.  Encouraged  by  the  cost-effective 
advantages  gained  through  the  Acoustic  Rapid  commercial  off-the-shelf 
Insertion/Advanced  Processor  Build  (ARCI/APB)  program,  Admiral  Mullen  noted  in  a 
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recent  memo  his  disappointment  with  the  slow  pace  of  adoption  and  advocated  rapid 
transition  to  the  open-business  model  (Fein,  2006,  September  11). 

D.  SA:  Track-management  Processes 

Track  management,  a  fundamental  capability  inherent  to  all  IWS  SA 
capabilities  for  surface  ships,  is  the  process  by  which  friendly  and  enemy  forces  are 
detected,  identified,  monitored,  updated  and  communicated  throughout  the  area  of 
operations  (AOR).  The  track  management  process  within  a  Combat  Information 
Center  (CIC)  is  very  complex,  sophisticated  and  involves  multiple  watch  stations  and 
technological  systems.  AEGIS  and  SSDS  have  different  SA  procedures  and 
policies,  and  track-management  functions  within  the  CIC.  Although  variations  exist 
in  track-management  processes,  watch  stations  are  fairly  consistent  on  both  AEGIS 
and  SSDS  ships.  Figure  7  is  a  generalized  organizational  chart  of  CIC  personnel 
directly  involved  in  track-management  processes.7 


Figure  7.  CIC  Organizational  Chart 


Track-management  processes  entail  various  sub-processes,  as  seen  in 
Figure  8.8  This  graphic  is  an  aggregated  view  for  both  AEGIS  and  SSDS  platforms 
consisting  of  four  principal  processes  and  17  sub-processes. 


Although  watch  stations  talk  to  specific  tasks  and  responsibilities,  in  an  actual  CIC,  all  personnel  listed  can  be  actively 
involved  in  any,  or  all,  aspects  of  track  management  (correlation,  identification,  tracking,  and  relaying). 

Figure  derived  from  numerous  SMEs  from  AEGIS  and  SSDS  communities.  While  sub-processes  may  differ  from  ship  to 
ship,  SMEs  concluded  that  the  four  primary  processes  reflect  track  management  procedures  conducted  within  a  CIC. 
SMEs  agreed  that  the  17  sub-processes  shown  reflected  individual  tasks  appropriate  for  this  limited  research.  SMEs  also 
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Figure  8.  Track  Management  Sub-processes 
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concluded  that  there  is  no  definitive  sequential  order  in  which  specific  tasks  occur;  however,  the  figure  provides  a  potential 
sequence. 
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VI.  Case  Study  Results:  Potential  Impact  of  OA 


The  potential  impact  of  OA  on  AEGIS  and  SSDS  platforms  was  calculated 
through  KVA+RO  in  a  multi-phased  approach.  In  addition,  KVA  data  estimates  and 
real  options  models  were  based  on  the  assumption  that  functional  upgrades  would 
primarily  be  in  the  SA  area  with  the  remaining  modules  providing  at  least  the  same 
capabilities  as  the  current  AEGIS  IWS  system. 

KVA  methodology  was  first  applied  to  derive  potential  benefits  in  SA 
processes  within  AEGIS  and  SSDS-class  ships  (i.e. ,  84  Destroyers  and  Cruisers). 
Track-management  sub-processes  by  process  category  (and  by  watch  station  in 
Appendix  B)  were  evaluated  under  four  improvement  scenarios  (“As  Is,”  “To  Be,” 
“Radical  1,”  and  “ Radical  2”).  The  following  assumptions  were  used  to  calculate  the 
data: 

■  Integration  with  middleware  until  Category  3  Open  Architecture 
Computing  Environment  (OACE)  level  has  been  reached  for  systems 
being  evaluated 

■  Use  of  OA  approach  to  developing  the  IWS  systems  and  use  of 
Commercial-off-the-shelf  (COTS)  equipment 

Steps  in  calculating  KVA  data  were: 

1 .  Identify  core  processes  and  sub-processes. 

2.  Establish  common  units  and  level  of  complexity  to  measure  learning 
time. 

3.  Calculate  learning  time  (i.e.,  knowledge  surrogate)  to  execute  each 
sub-process. 

4.  Designate  sampling  time  period  long  enough  to  capture  representative 
sample  of  the  core  processes’  final  product  or  services  output. 

5.  Multiply  learning  time  for  each  sub-process  by  number  of  times  sub¬ 
process  executes  during  sample  period. 

6.  Calculate  cost  to  execute  knowledge  (learning  time  and  process 
instructions)  to  determine  process  costs. 
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7.  Calculate  ROK  (ROK=  Revenue/Cost)  and  ROI  (ROK=  Revenue- 
Cost/Cost). 

During  Phase  2,  real  options  analysis  focused  on  the  options  for  improving 
(Leave  and  Layer  option)  or  replacing  ( Retrofitting  option)  the  AEGIS  IWS  system. 
The  option  to  continue  with  the  current  proprietary  systems  approach  was  provided 
as  a  baseline  for  comparison  purposes.  Future  research  would  allow  us  to  examine 
all  AEGIS  modules  for  potential  upgrading  and  would  likely  result  in  even  higher  ROI 
estimates  as  well  as  real  options  valuations. 

A.  KVA  Results:  ROK  and  ROI 

ROK  and  ROI  values  provide  insights  into  sub-processes  that  could  be 
reengineered  to  achieve  maximum  operational  efficiency.  Aggregated  results  for 
AEGIS  and  SSDS  are  shown  Figures  9  and  10.  The  “As  Is”  provides  a  baseline 
ROI/ROK  performance  measure  for  comparison  of  the  three  process  reengineering 
designs.  The  three  redesigns  essentially  represent  the  effects  of  increasing  levels  of 
automation  in  the  track  management  process. 
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Figure  9.  KVA  Results:  ROK  and  ROi  Estimates 


AEGIS 


Figure  10.  KVA  Results:  ROK  and  ROI  Estimates 


SS  DS 


OA  has  the  potential  to  provide  these  operational  performance  improvement 
benefits:  decreased  training  time  for  operators  of  systems,  decreased  “touch  time” 
on  processes  by  replacing  manual  processes  with  new  automated  capabilities,  and 
increased  efficiency  through  seamless  integration  of  multiple  system  components. 
As  shown  in  Table  2  below,  the  cumulative  impact  of  OA  on  the  track-management 
processes  results  in  significant  improvement  in  three  of  four  areas. 
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Table  2.  Process  Reengineering  Impacts:  Process  Level  and  Cumulative 


“To  Be” 

“Radical  1” 

“Radical  2” 

Cumulative 

Impact 

CORRELATE 

X 

X 

X 

TRACK 

X 

X 

X 

X 

IDENTIFY 

X 

X 

X 

X 

RELAY 

Tables  3  through  6  are  ROK  and  ROI  results  by  core  processes  and  sub¬ 
processes  for  AEGIS  and  SSDS.  The  ROI  estimates  demonstrate  that  as  various 
system  functionalities  of  the  existing  track-management  process  are  upgraded,  a 
corresponding  performance  improvement  is  derived  in  those  areas.  This  more 
detailed  analysis  suggests  where  OA  based  system  upgrades  should  be  applied  to 
achieve  the  best  results.  For  example,  the  core  process  of  correlate  and  identify 
have  the  greatest  potential  to  benefit  from  an  OA  approach  to  system  development. 
These  estimates  also  provide  the  basis  for  the  real  options  analysis  projections 
staged  over  a  9-year  period. 
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Table  3.  Detailed  ROK  Estimates  for  AEGIS 


AS  IS 

TO  BE 

RAD  1 

RAD  2 

CORRELATE 

Obtain  Link  Information 

3521% 

3407% 

3163% 

2733% 

Identify  "Same  Contact,  Multiple  Track" 

9% 

9% 

2161% 

1856% 

Verify  Other  Track  Sources 

5% 

5% 

5% 

4% 

Correlate  sub-total 

1284% 

1241% 

1606% 

1396% 

TRACK 

Monitor  Suspect  Tracks 

2% 

1% 

1% 

1% 

Update  Tracks 

3% 

3% 

461% 

410% 

Update  GCCS-M 

3% 

191% 

184% 

169% 

Track  sub-total 

2% 

6% 

42% 

36% 

IDENTIFY 

Verify  IFF  signal 

902% 

869% 

806% 

107% 

Verify  EW  emissions 

9% 

9% 

8% 

609% 

Verify  Point  of  Origin 

2% 

4221% 

3921% 

3432% 

Match  Against  ATO 

2% 

4306% 

3990% 

3482% 

Match  Against  CommAir  Profile 

963% 

935% 

863% 

743% 

Match  Against  Intel  Information 

3% 

3% 

3% 

3914% 

Examine  Kinematic  Data 

4% 

4% 

3% 

3% 

Obtain  Visual  ID 

0% 

0% 

0% 

0% 

Conduct  Verbal  Query 

1% 

1% 

1% 

1% 

Identify  sub-total 

108% 

160% 

150% 

426% 

RELAY 

Send  Over  Links 

13% 

12% 

11% 

10% 

Discuss  Picture  with  Battle  Force  Units 

2% 

1% 

1% 

1% 

Relay  sub-total 

3% 

2% 

2% 

1% 

TOTALS 

312% 

340% 

373% 

504% 

Table  4.  Detailed  ROK  Estimates  for  SSDS 


AS  IS 

TO  BE 

RAD  1 

RAD  2 

CORRELATE 

Obtain  Link  Information 

3493% 

3380% 

3126% 

2698% 

Identify  "Same  Contact,  Multiple  Track" 

9% 

9% 

2630% 

2258% 

Verify  Other  Track  Sources 

5% 

5% 

4% 

4% 

Correlate  sub-total 

1274% 

1231% 

1612% 

1401% 

TRACK 

Monitor  Suspect  Tracks 

2% 

1% 

1% 

1% 

Update  Tracks 

4% 

3% 

646% 

575% 

Update  GCCS-M 

2% 

114% 

110% 

101% 

Track  sub-total 

2% 

4% 

47% 

40% 

IDENTIFY 

Verify  IFF  signal 

890% 

857% 

792% 

695% 

Verify  EW  emissions 

10% 

9% 

9% 

574% 

Verify  Point  of  Origin 

2% 

3789% 

3505% 

3067% 

Match  Against  ATO 

2% 

3913% 

3610% 

3149% 

Match  Against  CommAir  Profile 

1026% 

996% 

916% 

788% 

Match  Against  Intel  Information 

3% 

3% 

3% 

3788% 

Examine  Kinematic  Data 

4% 

4% 

4% 

4% 

Obtain  Visual  ID 

0% 

0% 

0% 

0% 

Conduct  Verbal  Query 

1% 

1% 

1% 

1% 

Identify  sub-total 

112% 

159% 

148% 

416% 

RELAY 

Send  Over  Links 

18% 

17% 

16% 

14% 

Discuss  Picture  with  Battle  Force  Units 

1% 

1% 

1% 

1% 

Relay  sub-total 

3% 

2% 

2% 

2% 

Totals 

312% 

337% 

371% 

499% 

-25- 


Table  5.  Detailed  ROI  Estimates  for  AEGIS 


AS  IS 

TO  BE 

RAD  1 

RAD  2 

CORRELATE 

Obtain  Link  Information 

3421% 

3307% 

3063% 

2633% 

Identify  "Same  Contact,  Multiple  Track" 

-91% 

-91% 

2061% 

1756% 

Verify  Other  Track  Sources 

-95% 

-95% 

-95% 

-96% 

Correlate  sub-total 

1184% 

1141% 

1506% 

1296% 

TRACK 

Monitor  Suspect  Tracks 

-98% 

-99% 

-99% 

-99% 

Update  Tracks 

-97% 

-97% 

361% 

310% 

Update  GCCS-M 

-97% 

91% 

84% 

69% 

Track  sub-total 

-98% 

-94% 

-58% 

-64% 

IDENTIFY 

Verify  IFF  signal 

802% 

769% 

706% 

607% 

Verify  EW  emissions 

-91% 

-91% 

-92% 

509% 

Verify  Point  of  Origin 

-98% 

4121% 

3821% 

3332% 

Match  Against  ATO 

-98% 

4206% 

3890% 

3382% 

Match  Against  CommAir  Profile 

863% 

835% 

763% 

643% 

Match  Against  Intel  Information 

-97% 

-97% 

-97% 

3814% 

Examine  Kinematic  Data 

-96% 

-96% 

-97% 

-97% 

Obtain  Visual  ID 

-100% 

-100% 

-100% 

-100% 

Conduct  Verbal  Query 

-99% 

-99% 

-99% 

-99% 

Identify  sub-total 

8% 

60% 

50% 

326% 

RELAY 

Send  Over  Links 

-87% 

-88% 

-89% 

-90% 

Discuss  Picture  with  Battle  Force  Units 

-98% 

-99% 

-99% 

-99% 

Relay  sub-total 

-97% 

-98% 

-98% 

-99% 

Totals 

212% 

240% 

273% 

404% 

Table  6.  Detailed  ROI  Estimates  for  SSDS 


AS  IS 

TO  BE 

RAD  1 

RAD  2 

CORRELATE 

Obtain  Link  Information 

3393% 

3280% 

3026% 

2598% 

Identify  "Same  Contact,  Multiple  Track" 

-91% 

-91% 

2530% 

2158% 

Verify  Other  Track  Sources 

-95% 

-95% 

-96% 

-96% 

Correlate  sub-total 

1174% 

1131% 

1512% 

1301% 

TRACK 

Monitor  Suspect  Tracks 

-98% 

-99% 

-99% 

-99% 

Update  Tracks 

-96% 

-97% 

546% 

475% 

Update  GCCS-M 

-98% 

14% 

10% 

1% 

Track  sub-total 

-98% 

-96% 

-53% 

-60% 

IDENTIFY 

Verify  IFF  signal 

790% 

757% 

692% 

595% 

Verify  EW  emissions 

-90% 

-91% 

-91% 

474% 

Verify  Point  of  Origin 

-98% 

3689% 

3405% 

2967% 

Match  Against  ATO 

-98% 

3813% 

3510% 

3049% 

Match  Against  CommAir  Profile 

926% 

896% 

816% 

688% 

Match  Against  Intel  Information 

-97% 

-97% 

-97% 

3688% 

Examine  Kinematic  Data 

-96% 

-96% 

-96% 

-96% 

Obtain  Visual  ID 

-100% 

-100% 

-100% 

-100% 

Conduct  Verbal  Query 

-99% 

-99% 

-99% 

-99% 

Identify  sub-total 

12% 

59% 

48% 

316% 

RELAY 

Send  Over  Links 

-82% 

-83% 

-84% 

-86% 

Discuss  Picture  with  Battle  Force  Units 

-99% 

-99% 

-99% 

-99% 

Relay  sub-total 

-97% 

-98% 

-98% 

-98% 

Totals 

212% 

237% 

271% 

399% 
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Table  7  summarizes  how  OA  could  specifically  impact  the  sub-processes 
deriving  most  significant  improvements:  “Identify  ‘Same  Contact,  Multiple  Track',” 
“Update  GCCS-M,”  “Update  Track,”  “Verify  Point  of  Origin,”  “Match  against  ATO,” 
“Verify  EW  Emissions”  and  “Match  Against  Intel  Information.”  Each  watch  station  at 
the  CIC  was  affected  (see  Appendix  B). 

Table  7.  Potential  Impact  of  OA  at  Sub-process  Levels 


PROCESS 

SUB-PROCESS 

COMMENTS  POTENTIAL  IMPACT  OF  OA 

CORRELATE 

“Identify  ‘Same 
Contact,  Multiple 
Track’” 

•  Reduces  reliance  on  manual  identification  of  multiple  tracks  and  updating  current  tracks. 

•  Automatically  corrects  anomaly  of  multiple  tracks  per  target  and  update  tracks.  Only  brief 
confirmation  by  the  watch  station  operator  necessary. 

TRACK 

“Update  GCCS-M” 

•  Enhances  operational  value  of  systems  through  reduced  time,  manpower  and  training 
required  to  conduct  process. 

TRACK 

“Update  Track” 

•  Reduces  reliance  on  manual  identification  of  multiple  tracks  and  updating  current  tracks. 

•  Automatically  corrects  multiple  tracks  per  target  anomaly  and  update  tracks,  resulting  in 
brief  confirmation  by  watch  station  operator. 

IDENTIFY 

“Verify  Point  of 
Origin” 

•  Enables  greater  sensor  and  data  integration,  providing  enhanced  correlation  in 
pinpointing  origin  of  aircraft  or  ship. 

•  Queries  point  of  origin  for  friendly  force  contacts  from  an  open  GCCS-M  system,  and 
interrogates  ATO  neutral-force  contacts  from  host  nation  airports  (assuming  data  format 
standardized  and  provided  by  host  nations.). 

•  Facilitates  interfaces  to  other  systems  to  provide  automated  query  for  point  of  origin. 

•  Frees  watch  standers  to  perform  other  tasks  while  providing  faster  data  flow. 

IDENTIFY 

“Match  Against 
ATO” 

•  Integrates  info  provided  in  ATO  into  the  AEGIS  and  SSDS  platforms,  greatly  reducing 
manpower  requirements. 

IDENTIFY 

“Verify  EW 
Emissions” 

•  Facilitates  COTS-based  environment  for  easier  upgrades  to  accommodate  greater 
processor  speeds. 

•  Enhances  CIC  efficiency  through  more  timely  SA. 

•  Frees  operators  to  perform  other  tasks. 

IDENTIFY 

“Match  Against  Intel 
Information” 

•  Streamlines  sub-process  with  automatic  updates  requiring  merely  manual  confirmation. 

B.  Value-risk  Analysis:  Strategic  Real  Options  Analysis 

Real  options  analysis  was  performed  to  determine  the  prospective  value  of 
alternative  COAs  for  upgrading  the  AEGIS  IWS  in  track  management  over  a  nine- 
year  period  with  KVA  data  inputs.  In  all  new  options  for  IWS  deployment,  it  was 
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assumed  that  a  collaborative  technology  infrastructure  was  present  to  facilitate  the 
use  of  the  OA  system  and  open-business  model  approaches.9 

Figure  1 1  illustrates  the  three  main  strategies  laid  out  as  a  real  options  map. 
Strategy  A  is  do  nothing,  leaving  everything  “As  Is”  with  the  ability  to  retire  ships  and 
their  AEGIS  systems  within  10  years.  Strategy  B  is  the  “DDX  Open  Architecture” 
(retrofit)  option  with  new  development  within  the  first  three  years  at  a  cost  of  $8 
billion.  Under  this  strategic  path,  follow-up  is  required — with  retrofitting  costing  an 
additional  $16  billion  within  6  years  after  first-phased  new  development  has  been 
completed.  Strategy  C  looks  at  the  “Leave  and  Layer”  option  with  a  three-phased 
sequential  compound  option  of  “To  Be,”  “  Radical  1,”,  and  “Radical  2” 
implementation  within  9  years. 


Estimates  are  based  on  historical  data  and  additional  information  provided  by  SMEs.  We  have  attempted  to  be  as 
conservative  as  possible  and  have  assumed  very  high  potential  volatility  in  both  of  the  new  IWS  development  options. 
Access  to  more  precise  performance  data  will  help  resolve  uncertainties  and  risks  over  time. 


-28- 


Figure  11.  Strategic  Real  Options  Map 


Start 


- ►  Strategy  A  — 

AS-IS  strategy  where  the  older 
ships  are  retired  within  1 0 
years  and  no  further 
development  action  is  taken. 
Option  Value  =  $12B 
Total  Cost  =  $10B 


- ►  Strategy  B  — 

DDX  Open  Architecture 
approach  of  new  development 
within  3  years  followed  by 
retrofitting  within  6  years,  with 
an  option  to  exit  and  abandon 
before  the  retrofitting  phase. 
Option  Value  =  $23.2B 
Total  Cost  =  $24B 


- ►  Strategy  C  — 

AEGIS  Open  Architecture  with 
a  Leave  &  Layer  approach, 
followed  by  a  9-year  sequential 
option  of  stage-gate  investment 
of  To-Be,  Radical,  and  Radical 
II,  with  the  options  to  exit  and 
abandon  at  any  phase. 

Option  Value  =  S58.8B 


Retire  Ships 
Within  10  years 


-►  Exit 


Do  nothing  for  now 


New 

Development 

Inv  =  $8B 
Within  3  years 


-►  Exit 
Do  nothing 


Leave  & 
Layer 


>  Exit 

Do  not  continue 
development 


-  Retrofitting 
Inv  =  $  1 6B 

Anytime  within  6  years 


-►  Exit 


Stop  after  new  development 
without  retrofitting 


Do  not  continue 
development 


Total  Cost  =  S9.09B 


-►  RADICAL  II 

Inv  =  $3.  IB 
Anytime  within  9  years 
(after  Radical) 


-►  Exit 

Do  not  continue 
development 


Strategy  C  is  the  option  with  the  greatest  potential  value.  As  seen  in  Table  8 
below,  Strategy  C  provides  4.9  times  the  risk  adjusted  return  of  the  “As  Is”  strategy 
versus  Strategy  B  at  1 .9  times  the  return.  Strategy  C’s  incremental  approach  offers 
the  lowest  risk  and  numerous  benefits,  including  less  disruption  to  the  rest  of  system 
and  deriving  benefits  faster.  It  is  the  lowest  total  cost  alternative  with  costs  spread 
over  a  nine-year  period,  yet  the  program  reaps  incremental  benefits  from  various 
functionality  improvements  throughout  that  time.  This  strategy  provides  the  highest 
NPV. 
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Table  8.  Real  Options  Valuation  Results:  Strategies  A-C 


Strategy  A 

Strategy  B 

Strategy  C 

STRATEGIC  OPTION 

“As  Is” 

DDX  OA 

“Develop  and  Retrofit” 

AEGIS  OA 
“Leave  &  Layer” 

Net  Present  Value 

$12B 

$6.38B 

$27.52B 

Volatility 

0% 

80.5% 

86.3% 

Real  Options  Value 

$12B 

$23.155B 

$58.84B 

Strategic  Real  Options- 
based  Return  on  Investment 

N/A 

72.36% 

224.75% 

Total  Cost 

$10B 

$24B 

$9.09B 

Strategic  Real  Options- 
based  Relative  Return  Ratio 

1.0 

1.9 

4.9 

Notes: 


(1 )  The  volatility  measure  quantifies  uncertainty  and  risk  levels  in  the  strategy  and  calibrates  them  to  account 
for  the  time  required  to  complete  the  entire  strategy. 

(2)  Strategic  real  options  values  are  also  computed,  accounting  for  value  of  open  architecture  as  laid  out  in 
the  phased-gate  development  process. 

(3)  Strategic  real  options-based  return  on  investment  looks  at  real  options  valuation  results  and  computes  ROI 
based  on  the  option  values  and  implementation  of  costs  each  phase.  The  higher  this  ROI  value,  the  more 
strategic  and  valuable  or  profitable  a  project. 

“Leave  and  Layer”  allows  organizations  to  benefit  from  incremental  adoption. 
Rather  that  executing  a  plan  that  requires  everything  to  be  accomplished  at  once, 
“leave  and  layer”  enables  existing  systems  to  be  reused  successfully.  NAVFAC 
successfully  adopted  the  approach  to  provide  a  more  efficient,  lower  cost  contract 
management  solution.  NAVFAC  architected  a  technology  platform  to  allow  it  to 
build  a  layer  of  Web-based  collaborative  project  management  tools,  while  leveraging 
existing  financial,  HR,  and  scheduling  systems.  A  number  of  applications  were 
developed,  including  the  collaborative  eProjects  application  with  a  budget  of 
$350,000  and  completion  in  10  months  (Oracle,  2004).  eProjects  provides  one-click 
schedule  and  cost  status.  Another  application,  eContracts,  automates  nearly  200 
redundant  screens  per  contract  action. 

In  our  analysis,  Strategy  B  has  the  highest  cost  due  to  high  up-front  costs 
required  to  build  the  system  within  the  first  five  years  without  deriving  any  benefits 
from  the  new  system  during  that  time.  Both  strategies  B  and  C  require  the  use  of 
collaborative  infrastructure  to  enable  the  open  business  model  that  would  be  most 
likely  to  produce  these  real  options  values.  Strategy  C,  in  fact,  relies  most  heavily 
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on  collaboration  to  enable  the  kinds  of  benefits  in  rapid,  spiral  acquisition  with 
greater  competition  and  innovation  from  smaller  players. 

Summary 

IWS  systems  that  were  developed  in  a  closed,  proprietary  model  have 
performed  well  and  provide  substantial  returns.  However,  a  new  paradigm  is 
required  to  maintain  military  superiority  and  wage  information-age  warfare.  Through 
open-system  development  and  open-business  models,  benefits  such  as  reusable 
code,  lower  maintenance-upgrade  costs,  and  greater  vendor  flexibility  in  supporting 
system  modules  could  be  derived.  Moreover,  the  Navy  can  leverage  new 
technology  by  quickly  adopting  it  to  military  needs. 

Significant  investments  are  required  for  the  infrastructure  necessary  to  enable 
all  parties  (acquirers,  users,  developers)  to  collaborate  easily  and  effectively  in  the 
new  open-business  model.  Analytical  tools  are  also  required  to  track  performance  of 
the  multiple  parties  involved  in  the  development,  acquisition  and  use  of  new  system 
capabilities,  in  conjunction  with  the  ability  to  adjust  options  models  as  uncertainties 
and  risks  are  resolved  over  time.  Performance  measurement  systems  (i.e.  KVA 
performance  accounting  software)  and  predictive,  forecasting  software  programs 
plus  risk  certification  training  provide  additional  analytic  support  to  achieve  IWS 
systems  acquisition  strategies. 
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Appendix  A.  Case  Study  Notes 


1.  Methodology 

Case  Study  Approach: 

The  learning-time  approach  to  KVA  is  used  to  conduct  sub-process  scenario 
analyses  of  “As  Is,”  “Radical  1,”  and  “Radical  2.”  Core  elements  of  KVA  (such  as 
“time-to-learn,”  “number  of  personnel  involved,”  and  “times  fired”)  produce  a  ratio  of 
knowledge  capital  (ROK)  resident  in  each  process.  ROK  derives  a  common  unit  of 
measurement  for  each  sub-process  within  the  track-management  process. 

Data  Collection: 

Collecting  data  to  conduct  KVA  analysis  was  difficult  due  to  the  complex 
nature  of  track-management  procedures  in  the  CIC.  Outputs,  learning  time  and 
touch  time  of  the  many  sub-processes  that  comprise  the  entire  procedure  are  not 
generally  collected  or  retained.  Also,  training  times  and  required  OJT  are  targeted  at 
specific  watch  stations  rather  than  at  specific  processes  within  the  Navy. 
Consequently,  data  was  derived  through  numerous  interviews  with  SMEs  and  review 
of  Personal  Qualification  Standards  (PQS).  Multiple  SMEs  were  contacted  to  collect 
an  aggregated  sample. 

“To  Be”  Data: 

Analysis  based  of  situations  that  SMEs  identified  as  optimal  areas  where 
open  architecture  could  provide  value  to  the  operator.  In  addition,  technical  and 
legal  issues  of  the  “To  Be”  scenario  were  not  assessed. 
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2.  Discussion  of  Basic  Assumptions  Used  in  Calculations 

Cost  and  Revenue  Data 

•  Calculations  based  on  a  ship  performing  SA  and  track  management 
processes  24  hours  a  day,  7  days  a  week  operating  an  average  of  35 
weeks  per  year. 

•  Net  present  value  in  total  revenue  estimates  is  based  on  a  30-year 
system-life  expectancy  of  ship. 

•  A  contractor-margin  approach  was  used  to  generate  surrogate 
comparable  revenue  estimates.  Contractor  margin  is  defined  as  the 
amount  market  place  would  pay  a  group  of  contractors,  with  levels  of 
knowledge  comparable  to  the  existing  team,  to  perform  the  activities  of 
the  track  management  team  (e.g  the  margin  over  current  Navy  costs 
for  the  track  management  team).  This  market-comps  approach  was 
used  because  there  were  no  commercial  processes  directly 
comparable  to  track  management  activities  in  the  military.  Future 
studies  would  allow  a  wider  range  of  potential  commercially 
comparable  processes  to  be  assessed  for  comparability/fit  revenue 
estimates. 

•  Surrogate  comparable  revenue  estimates  are  conservative  given  that 
revenue  calculations  were  based  on  six  people  in  the  SA,  track- 
management  process.  In  reality,  more  people  may  be  involved  in  this 
process. 

New  Development/Retrofitting  DD(x) 

•  PV  Asset  New  Development:  Calculated  by  translating  hourly  revenue 
of  the  combined  SSDS/AEGIS  “AS-IS”  estimates  to  yearly  revenue, 
calculating  the  present  value  of  the  yearly  revenue  based  on  a  30-year 
lifecycle,  and  assuming  retrofitting  on  all  current  84  Aegis 
destroyers/cruisers. 

•  PV  Asses  Retrofitting:  Calculated  by  translating  hourly  revenue  of  the 
combined  SSDS/AEGIS  RADICAL  estimates  to  yearly  revenue, 
calculating  the  present  value  of  the  yearly  revenue  based  on  a  30-year 
lifecycle,  and  assuming  retrofitting  on  all  current  84  AEGIS 
destroyers/cruisers. 

•  Cost  to  Execute:  Based  on  estimation  from  SMEs  and  refining  based 
on  historical  costs  of  AEGIS.* 

•  Operations/Maintenance  Costs:  Based  on  historical  costs  of  AEGIS 
and  scaled  down  to  account  for  open  architecture. 


-34- 


•  Timing:  Based  on  estimates  by  SMEs. 

Leave  and  Layer/AEGIS 

•  PV  Asset:  Calculated  by  translating  hourly  revenue  of  the  combined 
SSDS/AEGIS  estimates  to  yearly  revenue,  calculating  the  present 
value  of  the  yearly  revenue  based  on  a  30-year  lifecycle,  and  taking 
into  account  current  placement  on  84  destroyers/cruisers. 

•  Cost  to  Execute:  Based  on  $5  billion  development  costs  and  scaled 
according  to  increases  in  knowledge  units. 

•  Operations/Maintenance  Costs:  Based  on  historical  costs  of  AEGIS 
and  scaled  down  to  account  for  open  architecture. 

•  Timing:  Based  on  estimates  by  SMEs. 
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3.  Use  of  Data  in  Case  Study  Calculation 


Item 

Definition 

Comments 

No.  of 

Personnel 

Sailors  and  officers  involved 
in  performing  sub-process. 

— 

Actions  per 

Hour 

Times  each  sub-process 
acted  upon  by  watch- 
stander. 

•  Actions  predicated  on  amount  of  contacts  (air  and  surface)  encountered  during  a 
typical  hour  within  the  CIC.  Each  contact  must  be  acted  upon. 

•  Estimates  based  on  a  typical,  six-month  deployment. 

•  Number  of  contacts  based  on  average  of  open-ocean  transit  and  operations  in 
littorals. 

Actual  Work 

Time  (AWT) 

Specific  amount  of  time 
required  to  accomplish  action 
every  time  sub-process  acted 
upon. 

•  Data  captured  in  hourly  units. 

•  Although  actions  require  only  seconds,  category  captures  data  in  hours  to 
maintain  continuity  of  units  of  time  throughout  analysis. 

Total  Work 

Time 

Total  amount  of  time  each 
sub-process  acted  upon 
within  an  hour. 

•  Formula =  "AWT”  x  “Actions  per  Hour" 

•  Analysis  in  hourly  units:  when  “Total  Work  Time”  for  each  of  sub-processes 
added  together  for  each  of  the  watch  stations,  total  aggregate  should  remain 
below  1 .0.  If  total  exceeded  1 .0,  calculations  are  incorrect. 

Actual  Learning 
Time  (ALT) 

Total  amount  of  time  required 
to  learn  given  sub-process. 

•  Learning  time  can  be  an  aggregate  of  formal  schools,  distance  learning,  on-the- 
job  training  (OTJ)  or  any  other  training  experience  that  falls  under  definition  of 
“learning.” 

•  For  this  case,  comprised  of  formal  school  training  and  OJT  provided  aboard  ship. 

•  Basic  assumptions  to  ensure  consistent  estimates  from  SMEs: 

a.  Officer-SSDS 

Individual  completing  initial  officer  training  with  no  prior  SSDS  platform  experience.  It 
was  also  necessary  to  determine  formal  schools  represented  by  this  category.  While 
each  school’s  duration  is  considerably  longer  than  hours  represented  in  the  “ALT” 
category,  estimates  based  on  the  aggregated  amount  of  time  devoted  to  teaching 
given  sub-process  from  each  school:  SSDS  Basic  Operator  Course  of  Instruction, 
SSDS  Advanced  Operator  Course  of  Instruction,  and  SSDS  Warfare  Operator 

Course  of  Instruction. 

b.  Officer-AEGIS 

Individual  completing  officer  training  with  no  prior  AEGIS  platform  experience.  It  was 
also  necessary  to  determine  formal  schools  represented  by  this  category.  While 
each  school’s  duration  is  considerably  longer  than  the  hours  represented  in  the 
“ALT"  category,  estimates  based  on  aggregated  amount  of  time  devoted  to  teaching 
the  given  sub-process  from  each  school:  AEGIS  Training  Course,  SWOS  TAO 

School,  and  TAO  Simulator  Training. 

c.  Enlisted-SSDS 

Individual  completed  boot  camp  with  no  prior  SSDS  platform  experience.  It  was  also 
necessary  to  determine  formal  schools  represented  by  this  category.  While  each 
school’s  duration  considerably  longer  than  hours  represented  in  the  “ALT”  category, 
estimates  determined  based  on  aggregated  amount  of  time  devoted  to  teaching  the 
given  sub-process  from  each  school:  OS  “A”  School,  SSDS  Basic  Operator  Course 
of  Instruction,  SSDS  Advanced  Operator  Course  of  Instruction,  and  SSDS  Warfare 
Operator  Course  of  Instruction  (E5  and  above). 

d.  Enlisted-AEGIS 

Individual  completed  boot  camp  with  no  prior  AEGIS  platform  experience.  It  was  also 
necessary  to  determine  formal  schools  represented  by  this  category.  While  each 
school's  duration  is  considerably  longer  than  hours  represented  in  the  “ALT” 
category,  estimates  based  on  aggregated  amount  of  time  that  devoted  to  teaching 
given  sub-process  from  each  school:  OS  “A”  School  and  AEGIS  Console  Operator 
Course. 

Rank  Order 

An  ordinal  ranking  of  sub¬ 
processes  provides  a  means 
to  ensure  the  “ALT”  estimates 
are  reliable  and  as  accurate 
as  possible. 

•  Allowing  SMEs  to  rank/order  each  of  the  sub-processes  (1  being  the  least 
complex),  outside  the  context  of  units  of  time,  facilitates  a  mathematical 
correlation  achieved  between  “Rank  Order”  and  “ALT"  categories. 

•  If  correlation  is  .80  or  higher,  “ALT”  numbers  can  be  considered  an  accurate 
reflection  of  the  sub-process’s  complexity. 

•  If  correlation  is  below  .80,  “ALT”  estimates  should  be  closely  scrutinized  and 
possibly  reevaluated  after  providinq  a  better  explanation  of  the  “ALT”  components 
to  the  SMEs. 

Percent 

Percent  of  automation  for 
each  sub-process. 

•  Captures  knowledge  embedded  within  the  IT  so  that  it  can  be  accounted  for  in 
later  calculations.  Automation  is  defined  as  the  amount  of  the  sub-process  that  is 

-36- 


Information 

Technology 

(%IT) 

performed  by  information  technology  systems  and  does  not  require  the  actions  of 
an  operator. 

•  Each  sub-process  is  represented  by  a  percentage  between  0  and  100.  A  number 
of  100%  indicates  sub-process  is  completely  automated  and  does  not  require  a 
watch  stander  to  accomplish  any  portion  of  the  task. 

•  If  number  is  0%,  no  automation  exists,  and  the  watch-stander  completes  the  entire 
sub-process  manually.  Numbers  falling  between  extremes  are  estimates  based 
on  SME  observations  and  experience. 

Total  Learning 
Time  (TLT) 

Provides  total  time  required 
to  learn  sub-process, 
including  that  learning  time 
which  is  resident  within  the 

IT  system. 

•  Determined  by  “Actual  Learning  Time”  by  the  “Percent  Information  Technology” 
category. 

•  Formula  =  ALT/(1-%IT). 

•  For  instance:  If  it  takes  2  hours  to  learn  a  system  that  is  50%  automated,  then  the 
total  learning  time  for  that  system  (to  include  the  learning  time  that  is  embedded  in 
the  system  itself)  would  be  4  hours. 

Numerator 

Revenue  generated  by 
knowledge  required  to 
perform  sub-process. 

•  Revenue  allocated  to  the  amount  of  knowledge;  amount  of  knowledge  resident  in 
sub-process. 

•  Formula=  “Number  of  Personnel"  x  "Actions per  Hour"  x  “Total Learning  Time" 

Denominator 

Cost  associated  with 
producing  sub-process 
output. 

•  Formula  =  “Number  of  Personnel”  X  “Actions  per  Hour”  X  “Actual  Work  Time” 

Return  on 

Knowledge 

(ROK) 

Represents  how  well 
knowledge  assets  in 
organization  are  distributed 
based  upon  cost  and  value 
each  provides. 

•  With  every  sub-process,  there  is  a  cost  and  revenue  (or  value)  associated  with 
generating  an  output.  While  these  costs  and  values  are  captured  in  the 
“Numerator”  and  “Denominator"  categories,  there  needs  to  be  a  way  to  quantify 
the  knowledge  embedded  within  an  IT  system. 

•  ROK’s  can  be  compared  within  a  process  to  help  determine  if  knowledge  assets 
are  being  used  in  an  efficient  manner;  if  automation  could  be  inserted  to  improve 
outputs;  and  if  processes  should  be  changed  to  promote  efficiencies. 

•  A  low  ROK  does  not  automatically  assume  a  process  is  inefficient  or  in  need  of 
automation,  but  rather  is  an  indicator  that  a  sub-process  may  need  further 
analysis  to  determine  if  it  is  using  its  knowledge  assets  in  an  efficient  manner. 

4.  Variability  Report 

Average  Work  Time:  5%  Variability — The  time  it  takes  to  complete  each  action 

is  relatively  stable  with  little  or  no  variability. 

Average  Learning  Time:  5%  Variability — Estimates  regarding  average  learning 

time  are  based  on  the  time  it  takes  an  average  person  to 
learn  each  task,  hence,  low  variability. 

Price:  Assume  60%  of  the  time  the  position  is  priced  at  the 

average  of  the  low  and  high  estimates.  The  remaining 
time  is  split  20-20  between  the  low  and  high  values.  A 
custom  distribution  is  utilized  to  fulfill  these  requirements. 


Watch  Station 


Cost  Range  (per  hour) 
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Tactical  Action  Officer  (TAO) 

Anti-air  Warfare  Coordinator  (AAWC) 
Surface  Warfare  Coordinator  (SUWC) 
Combat  Systems  Coordinator  (CSC) 
Tactical  Information  Coordinator  (TIC) 
Identification  Supervisor  (IDS) 


$85  to  $105,  $80.00  to  $11 0.00 
$75  to  $90,  $72.00  to  $92.00 
$75  to  $90,  $72.00  to  $92.00 
$78  to  $95,  $69.00  to  $89.00 
$70  to  $80,  $65.00  to  $85.00 
$70  to  $80,  $63.00  to  $83.00 


Note:  Prices  provided  by  commercial  vendors. 


Cost:  Assumes  60%  of  the  time  the  position  is  filled  by  a 

person  with  the  assumed  pay  grade.  The  remaining  time 
is  split  20-20%  between  a  person  with  one  rank  higher 
and  one  rank  lower.10  A  custom  distribution  is  utilized  to 
fulfill  these  requirements. 


Watch  Station 

Years  of  Service 

Cost 

Tactical  Action  Officer  (TAO):  (0-5) 

between  10-18 

$38 

Anti-Air  Warfare  Coordinator  (AAWC):  (0-4) 

between  8-16 

$34 

Surface  Warfare  Coordinator  (SUWC):  (0-3) 

between  6-14 

$30 

Combat  Systems  Coordinator  (CSC):  (E-8) 

between  10-18 

$27 

Tactical  Information  Coordinator  (TIC):  (E-6) 

between  4-14 

$17 

Identification  Supervisor  (IDS):  (E-5) 

between  3-8 

$14 

Note:  Costs  calculated  by  averaging  monthly  salary  plus  sea  pay  for  the  assumed 

ranks  at  low/high  estimates  of  years  in  service. 


Actions  per  Hour:  A  triangular  distribution  with  min/max/most-likely  values 

based  on  calculations  from  the  following  numbers. 


Costal 

Ooen  Water 

Hiah  Density 

Number  of  Contacts  per  Hour 

24-42 

12-30 

28-66 

Time  in  Location 

15% 

25% 

60% 

Note:  Data  provided  by  SMEs 


10  In  the  case  of  a  TAO,  assume  80%  of  the  time  the  position  is  filled  by  an  0-5;  in  the  case  of  an  IDS, 
assume  80%  or  the  time  the  position  is  filled  by  an  E-5. 
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Appendix  B.  KVA  Results  by  Watch  Station 


Figure  B-1.  AEGIS  ROI  Estimates 
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2230% 
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-91% 

-91% 

2104% 
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2983% 
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-95% 
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-92% 
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1358% 
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1791% 

1512% 
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-99% 
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-99% 
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- 

- 

- 

- 
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-89% 
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-91% 
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-92% 
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3450% 
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-97% 
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- 

- 
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- 
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- 

- 
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3813% 
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-96% 
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55% 
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596% 

84% 
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TOTAL 
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558% 
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400% 
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9% 
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Figure  B-2.  AEGIS  ROI  Estimates  (cont.) 
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_ 
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_ 
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-13% 
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2% 
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822% 
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1104% 

1074% 

982% 
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-96% 
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-85% 
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1030% 
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-99% 
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-99% 
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4146% 
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1178% 

1147% 

1053% 

962% 

1405% 

1368% 

1253% 

1142% 
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-99% 

-99% 

-99% 

1462% 

-96% 
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-95% 

-95% 
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-95% 
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Figure  B-3.  SSDS  ROI  Estimates 


Tactical  Action  Officer 
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Figure  B-4.  SSDS  ROI  Estimates  (cont.) 
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-98% 

253% 

187% 

-93% 

-94% 

1077% 

984% 

-92% 

-92% 

1282% 

1168% 

Update  GCCS-M 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

Track  Sub-Total 

-99% 

-99% 

-90% 

-92% 

-95% 

-95% 

75% 

61% 

-94% 

-94% 

105% 

88% 

IDENTIFY 

Verify  IFF  signal 

446% 

443% 

394% 

301% 

1735% 

1691% 

1548% 

1417% 

2062% 

2008% 

1834% 

1676% 

Verify  EW  emissions 

-92% 

-92% 

-93% 

356% 

-72% 

-73% 

-75% 

1624% 

-68% 

-68% 

-71% 

1918% 

Verify  Point  of  Origin 

-99% 

2613% 

2370% 

1907% 

-95% 

8856% 

8141% 

7487% 

-95% 

10441% 

9572% 

8779% 

Match  Against  ATO 

-99% 

2613% 

2370% 

1907% 

-95% 

8856% 

8141% 

7487% 

-95% 

10441% 

9572% 

8779% 

Match  Against  CommAir  Profile 

582% 

578% 

517% 

402% 

2194% 

2139% 

1960% 

1797% 

2602% 

2535% 

2318% 

2120% 

Match  Against  Intel  Information 

-98% 

-98% 

-98% 

2687% 

-92% 

-93% 

-93% 

10437% 

-91% 

-91% 

-92% 

12231% 

Examine  Kinematic  Data 

-97% 

-97% 

-98% 

-98% 

-91% 

-91% 

-92% 

-92% 

-89% 

-89% 

-90% 

-91% 

Obtain  Visual  ID 

- 

- 

- 

- 

-99% 

-99% 

-99% 

-99% 

-99% 

-99% 

-99% 

-99% 

Conduct  Verbal  Query 

- 

- 

- 

- 

-98% 

-98% 

-98% 

-98% 

-97% 

-97% 

-98% 

-98% 

Identify  Sub-Total 

-17% 

-9%o 

-17% 

346% 

118% 

203% 

179% 

512% 

257% 

406% 

364% 

977% 

RELAY 

Send  Over  Links 

-93% 

-93% 

-94% 

-95% 

-77% 

-78% 

-79% 

-81% 

-73% 

-74% 

-76% 

-78% 

Discuss  Picture  with  Battle  Force  Units 

- 

- 

- 

- 

-97% 

-97% 

-97% 

-97% 

-96% 

-96% 

-97% 

-97% 

Relay  Sub-Total 

-93% 

-100% 

-100% 

-100% 

-96% 

-100% 

-100% 

-100% 

-91% 

-91% 

-92% 

-92% 

TOTAL 

256% 

260% 

300% 

459% 

489% 

544% 

613% 

839% 

615% 

682% 

766% 

1040% 
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Appendix  C.  Real  Options  Analysis 


Table  C-2.  KVA  Analysis  with  Monte  Carlo  risk-based  simulations 
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File  Help 


lumple  Asset  buper  Lattice  bolver 

_  □  X 

Maturity 


Comment  Strategy  B 


Underlying  Assets 


Custom  Variables 


Name 


Underlying 


PV  Asset  Volatility  (%)  Notes 


Option  Valuations  - 

Blackout  and  Vesting  Period  Steps 


Name 
► ,Phase2 
Phase 1 

* 


Cost  Risk  Free...  Dividend...  Steps  Terminal  Equation 

UMtomsmmsmMsm 

3  5  0  50  Max(Phase2-Cost,  0) 


Name 

Value 

Starting  Step 

|  Expansion 

2.6296 | 

_ II 

Result 

Ready. 


Create  Audit  Sheet  |  Run 


Table  C-4.  Strategy  C:  Real  Options  Analysis  Results 
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